Event Visualization

ABSTRACT

The present invention includes a system and process to visualize an event. The event visualization includes one or more physical entities that generate data experienced thereby and generate executable files related to their vantage points. A file manipulation engines normalizes data both existing in pure data form as well as data within the executable files. A visual display renders a representation of the data along with executable files that are correlated to the data. A relationship between two or more physical entities may be determined, and multimedia files of one entity that includes the other may be determined. Entities with no relationship between them, other than physical proximity (or other relationship) at a particular time, may offer multimedia files to the other entity that includes that entity.

RELATED APPLICATIONS

This application is a continuation, and claims priority under 35 U.S.C.§120 from U.S. patent application Ser. No. 15/284,495 titled EventVisualization filed on Oct. 3, 2016.

FIELD OF THE INVENTION

The present invention relates to the field of data analysis and morespecifically to the field of computer-aided eventsimulation/reproduction.

BACKGROUND

Data analysis is more of an art than a science. Many events and actionsare foreseeable, however, the rate at which data can be processed andreviewed, and the staggering amount of data available, makes dataanalysis a formidable task. For any given day the data exists todetermine the weather for the next day, but that data is also hiddenamongst data that leads to the incorrect conclusion. The ability to drawvalid relationships between data points involves wisdom and experience,as much as good data. Nevertheless the data must be available, and datalacking significant information should be minimized whileinformation-rich data should be capable of simplified access.

Enormous advances have been made in processing, searching and extractingspecific information and items of interest from the ever-increasingamount of collected data. Piecewise datasets from many data sources thatare related to a specific event of interest are frequently identifiedand isolated for detailed review. These datasets, such as trackingcoordinates, images, videos, metadata, etc., each hold potentialcontributions in describing the event of interest from uniqueperspectives. Without the right toolset, organizing these datasets in amanner that ensures each impact to the event of interest can be realizedis a daunting task. Additionally, examining each data product separatelyrarely develops the necessary comprehensive understanding of how theloosely related datasets interrelate.

Constructing a visualization of event data is an effective tool forconceptualizing and sharing an understanding of what occurred, but itseffectiveness has been hampered by the time consuming nature of manualsynchronization and manipulation of datasets. Analysts reviewing eventdriven data need a comprehensive, state of the art, data synchronizationand visualization tool that allows them to interpret the eventsdescribed by assorted data; and, to create a meaningful visualrepresentation of event artifacts to share with collogues andmanagement.

Therefore, there is a need for process and system that allows data frommultiple points to be manipulated and compared in a meaningful way.Furthermore, the comparison should be capable of being guided by a humanoperator.

SUMMARY

The present invention is a synchronization and event visualization tooldesigned to mitigate common data aggregation challenges encounteredwhile processing data from disparate sources. It combines and organizesdata from various sources into a comprehensive visualization of therepresentative sequence of events. An interactive visual interfaceallows analysts to quickly navigate to key moments of each event,providing a detailed geospatial and temporal context for understandingthe event and directing further analysis. Additionally, the presentinvention's presentation production toolset and media synchronizationallows users to summarize and report their findings to decision makersin a clear and informative visual presentation.

The present invention is directed to an event visualization method andsystem. The system includes initializing a location data set of aphysical entity comprising location data points. The location datapoints contain location metadata constituents. Preferred metadataconstituents include geospatial indicia and time indicia. The methodinitializes a multimedia file from a vantage of the entity comprisingmultimedia metadata constituents.

A file manipulation engine compares metadata constituents of thelocation data points to metadata constituents of the multimedia file toascertain categorical metadata constituents. The engine normalizes thecategorical metadata constituents to ensure uniformity of data.

A visual display renders a sequential dimensional representation of thelocation data set changing as a function of at least one metadataconstituent. During the rendering step, a multimedia file that has beencorrelated to at least one of the location data points is initiated. Thecorrelation is based on at least one of the categorical metadataconstituents.

The event visualization method may initialize a first multimedia filefrom a first physical entity comprising multiple quantifiable firstmultimedia metadata constituents. A second multimedia file may beinitialized from a second physical entity comprising multiplequantifiable second multimedia metadata constituents. The quantifiablemetadata constituents of the multimedia files may be compared toascertain categorical quantifiable metadata constituents and normalizethe categorical quantifiable metadata constituents.

The visualization method may determine a domain of the quantifiablemetadata constituents to render on a visual display the first multimediafile and the second multimedia file in synchronization and based on asequential rendition of the first multimedia file and the secondmultimedia file further based on, and substantially equivalent to thedomain of, any selection of the multiple quantifiable categoricalmetadata constituents. A physical relationship is concluded based on aphysical relationship based on any selection of the multiplequantifiable categorical metadata constituents between the firstphysical entity and the second physical entity.

These aspects of the invention are not meant to be exclusive.Furthermore, some features may apply to certain versions of theinvention, but not others. Other features, aspects, and advantages ofthe present invention will be readily apparent to those of ordinaryskill in the art when read in conjunction with the followingdescription, and accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a view of the system of the present invention.

FIG. 2 is a view of the system of the present invention.

FIG. 3 is a view of the system of the present invention.

FIG. 4 is a view of the system of the present invention.

FIG. 5 is a view of the process of the present invention.

FIG. 6 is a view of the process of the present invention.

FIG. 7 is a view of the system of the present invention.

FIG. 8 is a view of the file manipulation engine of the presentinvention.

FIG. 9 is a view of the computer ecosystem of the present invention.

FIG. 10 is a view of the computer ecosystem of the present invention.

DETAILED DESCRIPTION

Referring first to FIG. 1, a basic embodiment of the event visualizationsystem 100 is shown. The system 100 includes one or more physicalentities 102 a, 102 b. The physical entity of the present inventionincludes any person or thing whose position may be desired to be trackedfor later event visualization. An example physical entity, and one thatwill be presently discussed, is a weather balloon. The physical entity102 a has a quantifiable relationship with another physical entity(e.g., an ancillary physical entity 102 b) or ecosystem 900. In theevent that a visualization of the physical entity 102 or the experiencesthereof is desired for later informational analysis, the experiences ofthe physical entity are retained. By experiences, it is meant theobservations of the physical entity 102 as they relate to (i)observations of the physical entity 102 as they relate to the attributesof the physical entity 102 itself, (ii) perceptions of the physicalentity 102 as they relate to the attributes of another specific entity,including an ancillary physical entity, and (iii) the perception of thephysical entity 102 with respect to its general surroundings. Thephysical entity may be any tangible object whose relevance may bedetermined on a case-by-case basis, including subsequent to an incidentfor which data analysis is beneficial. The physical entity need not beowned or managed by the source operating the present invention, andoften will not be; the present invention excels at aggregating faciallydisparate data types from disparate data sources into a single, combinedsimulation.

The physical entities 102 generate experiences through the use of adetection array 104. The detection array 104 includes the equipmentnecessary to generate the observations and perceptions described above.Preferred experiences include as either observations or perceptions:geolocation data, time indicia, wind speed, wind direction, pressure,velocity, altitude, temperate, humidity, acceleration, angle, torque,linear momentum, level, electrical readings, density, deformation,shape, radioactivity, audiovisual capture, measures of confidence,actions, communication (text, email, web browsing, etc.), electronicemissions, etc. Within the present specification, “observation” is meantto include measurement of phenomena related to the physical entity 102carrying the detection array 104; “perception” is meant to includemeasure of phenomena exterior to the physical entity 102 carrying thedetection array 104. The physical entity detection array carries one ormore instruments adapted to measure the above phenomena, such asthermometers, accelerometers, Global Positioning System instruments,barometers, etc. The sources of data frequently include detection arraysin the environment as well as mounted on other controlled anduncontrolled physical entities. In general, the invention makes noassumptions about how the data is collected. It provides the toolsnecessary to be able to visualize and organize the data forvisualization.

Turning now to FIG. 7, the physical entity detection array generatesinformation sufficient to create an entity location data set 140. Thelocation data set may be constructed as a formatted data file at thetime of creation, or the location data set may be created inpost-production as a result of normalizing and aggregating disparatedata types, or otherwise. The location data set 140 includes data points144 that include metadata constituents 146. That is to say the locationdata set 140 is arranged as a function of a single data type, whethertime, altitude, pressure, etc. As the nature of arrangement of data bytime represents a preferred embodiment, the data points 144 of FIG. 7are arranged by time. The data points are stored as a series ofquantities pursuant to which the metadata constituents are correlated.For example at time domain 1: the metadata constituents storedthereunder include Time: 1 s; Velocity: 10 m/s; Latitude: 37.6;Longitude −79.2; Video: vid.mp4; and pressure 101.2. As the data pointsprogress, the metadata correlated therewith is stored therewith. Themetadata constituents need not be constant; in other words, the metadataconstituent of longitude may be present in one data point but notanother. The metadata constituents need not be exactly correlated to thedata type representing the data point; in other words, at time Time: 1 sthere may be a velocity reading of 10 m/s while there is no velocityreading at Time: 2 s, yet there is a velocity reading at Time: 2.2 s of12 m/s. In such a case, the velocity element at Time: 2 s may beomitted, or estimated (for example, by plotting an average accelerationbetween the time points 1 s and 2.2 s and extrapolating the velocity at2 s), or other. The location data sets may be stored in table form, listform, or other data storage form. Although the data sets discussedherein as discussed as location data sets, this is because data setsrelating to a location are a preferred embodiment of the presentinvention; data sets may be categorized pursuant to any data type.

The present invention permits data relationships to be explored in anew, granular manner. A primary basis of the relationship exploration isthe linking of an executable file 142 in a file database 294 to a dataset 140 in a data set database 292. When FIG. 7 is viewed in conjunctionwith FIG. 5, the method 200 of the present invention initializes 202 thedata set 140. By initializing, it is meant to make an executable file ordata available for further processing. An executable file 142 isinitialized 204. A file manipulation engine within memory examines datato determine a data type of each set of data desired to be used with thepresent invention. For example, the file manipulation engine may lookfor velocity data types in multiple data sources to notate that the datafrom different files are categorically equivalent, irrespective ofmeasurement units (e.g., m/s, km/h, etc.) and file types (.xls, .csv,etc.). Furthermore, the file manipulation engine compares the data ofthe data sets 140 to any data that may be present in the executablefiles 142. By executable file, it is meant any file having data andcapable of running a software routine on instructions contained withinthe file (e.g., MS Word program), or any file capable of running asoftware routine on instructions contained within the file and iscapable of accessing data from an exterior file (e.g., browser sending aGET request based on a link), or any file having data preformatted tothe specifications of an identified file type for being run as asoftware routine on instructions contained within an exterior softwareprogram (e.g., jpg or mpg.). Examples of executable files includemovies, pictures, audio files, graphs, etc.

Executable files, especially movies, often have data elements associatedwith the data displayed therewith. Movies for example, may reference asinternal data: time, date, location, latitude, longitude, creator, etc.The file manipulation engine compares 206 data types internal to theexecutable file to data types of the data sets. The file manipulationengine may access a relational database 296 that correlates related datasets and provides conversion functions and other information helpful forcomparison and manipulation. The file manipulation engine in itscomparison is looking for data of similar categories, e.g. Time: Time,Velocity: Velocity, etc. In its analysis of file data, the filemanipulation engine may further normalize 208 the metadata.Normalization 208 under the present invention means checking to ensurethat the data can be meaningfully related together in a validcomparison, and includes one or more of the following actions: (i)checking to ensure that the data needs no normalization; (ii) ensuringthat data is appropriately parallel (e.g., to ensure that the dataoperates in relation to a common reference, i.e. a datum of 5 s for afile starting at 12:57:05 may be normalized to 12:57:10 when theappropriate reference is a timeline generally rather than a relativetime standard); (iii) to ensure that data is expressed in the same units(e.g., kg/hr to m/s); (iv) to ensure that data is expressed in the samelogical form (e.g., integer, decimal, character array); (v) to ensurethat data originating from difference sources is expressed with anexpected variance of measurement (e.g., an electronic emission detectedby two separate physical entities may have variance in the emissionfrequency detected). In certain embodiments of the present invention thenormalization 208 step may be the first step performed pursuant to amassive ingest process by which data is identified and normalized.

The data constituents of the executable files are correlated to the dataconstituents of one or more data sets. In other words, the applicabilityof an executable file based on its data constituents is discerned andthen noted. The preferred notation is incorporation into the data setitself as a pointer, however, the notation may be through a discretelist or direct insertion of the data into the data set. As shown in themetadata constituents 146 of FIG. 7, the date point no. 1 includes apointer to a video file, “vid.mp4.” Vid.mp4 was determined to beapplicable to the data point no. 1 just as “intel.docx” was determinedto be applicable to data point no. 3.

Returning to FIG. 1, along with reference to FIGS. 2-5 and 7, the datasets 140 created by the physical entity 102 a may be related to locationdata, and there may be a multimedia files 142. The multimedia files 142are stored in the file database 294 and the location data sets arestored in the location data set database 292 which may be created withinthe memory 110 of the physical entity 102 a. The physical entity createsdata, whether or not formatted as a structured data set of the presentinvention, as it undertakes activity. Here, the whether balloon 102 amay be released proximate to an object 900, i.e. a mountain or asecond/ancillary physical entity 102 b. While aggregating location datathe detection array may further include multimedia detection, forexample a camera that is taking pictures each second. The pictures maybe saved as multimedia files in the memory of the physical entity. Forthe purposes of the present disclosure, the location data set may becapturing the GPS coordinates of the physical entity inthree-dimensions. A preferred embodiment of the present inventiongenerates both an entity location data set comprising location datapoints bearing location metadata constituents, including geospatialindicia and time indicia, and multimedia files from a vantage of theentity comprising multimedia metadata constituents.

The system 100 includes a visual display 120 adapted to render asequential dimensional representation of the data set. The visualdisplay 120 includes a screen or other means for rendering visualelements. Examples of visual displays suitable for use with the presentinvention include mobile electronic communication devices, laptops,computers, televisions, and the like. The visual display 120 is insignaled communication with a computer 150, which may include anyarithmetic logic unit, capable of storing information in memory andprocessing program instructions. By sequential dimensionalrepresentation 116 of a data set, it is meant that there is a rendering210 of the magnitude of the data set constituent as a function of asecond data set constituent, and that depiction occupies at onedimension. In the weather balloon example, the altitude may berepresented as a function of time. As the weather balloons altitudechanges, the representation may rise and fall proportional to thealtitude of the weather balloon with a trace revealing previousaltitudes. In a preferred embodiment, an underlayer is fetched 216 froman underlayer database 298 to position as an underlay to therepresentation. In the weather balloon example, GPS coordinates may bedisplayed as a representation 116 as a function of time inthree-dimensions. An underlayer for such an embodiment may include amap, such as a satellite map, road map, pulse-doppler precipitation map,or topographical map. The position of the weather balloon, as thephysical entity, is rendered 210 in the representation 116 over thecourse of a time span.

The system 100 and method 200 further initiates 212 during the renderingstep 210 one or more executable files. As the data set, here a locationdata set, corresponding to geolocation data, is depicted as a functionof time, an executable file is correlated to the time data type of thelocation data set, for example as in the pointer correlation in datapoint no. 1 of FIG. 7. When the data set renders the data point to whichan executable file is correlated, the executable file is executed. Inthe case of a multimedia file, the multimedia file may bedepicted/played; and further in the case of the weather balloon example,a video recorder as the detection array may capture views of themountain (object 900) from the vantage point of the weather balloon. Byvantage point, it is meant the perception of one object from theperspective of a second object.

It is preferred that the executable files 114 a are initiated in adiscrete file window 112 with an indication (here, a dashed line) tomark the point (here, moment) whereby the multimedia file relates to thedata set. It is preferred that the program track the rendering of thedata set to position the file window in a way that prevents it frominterfering with the data set depiction. The executable file may notonly be initiated pursuant to a data point, but also concluded 214pursuant to metadata constituents within the executable file as may becorrelated to data points within a data set. For example, a movie thatbegins at Time(X)=14:30:25 and ends at Time(Y)=14:30:55 may becorrelated with a beginning pointer at Time(X) and an end pointer atTime(Y). The movie may be initiated at Time(X) and concluded at Time(Y);in the interval the movie plays in concert with the rendering of thedata set.

The weather balloon example is but an example of a logical use of thepresent invention. Furthermore, the present invention may render 210 adata set as a function of altitude. For example, if two survey drones102 a, 102 b as physical entities, as in FIG. 1, are dropped fromarbitrary heights above a mountain, the object 900. If the physicalfeatures of the mountain in multiple dimensions are the purpose of theventure, then the vantage of the physical entities as a function of timeis less appropriate than the vantage of the physical entities as afunction of altitude/height. As the parachute-aided drones are droppedand capture pictorial views of the mountain from their respectivevantage points, the location data sets are updated with the geolocationof the drones as stored in the data set database 292 and the multimediadepictions of the mountain as multimedia files stored in the multimediafile database 294. Once rendered, the location data set may be renderedas a function of altitude, which may include a representation 116 of apointed line descending at the rate of descent of the drone with apictorial depiction of the picture of the mountain at that altitude.Each drone 102 a, 102 b may have its distinct representation 116 a, 116b on the visual display 120 with its distinct depiction of companionmultimedia files 114 a, 114 b initiated 212 at the altitudecorresponding to the data point contemporaneously being rendered 210.

In a further embodiment of the present invention, the method of thepresent invention initializes a first multimedia file from a firstancillary physical entity. The multimedia file includes locationmetadata constituents in reference to the estimated location of a targetentity. A second multimedia file, from a second ancillary physicalentity is initiated and includes a multimedia executable file inreference to projected visual imagery of the target entity. Oncerendered, in synchronization and sequentially on the visual display,visual imagery of the target entity can be visualized in conjunctionwith the current location of the target entity.

An example of the previous embodiment can be given by considering thedata police may utilize to investigate a crime. After the police gainevidence that a suspect may have committed a crime. The police mayreceive a warrant and pull cell phone tower records from a major cellphone provider. The cell tower records may contain location data for thesuspect's cell phone during the incident time of the crime. Then theymay pull video surveillance data from various Department ofTransportation street cameras and nearby convenience stores. When theinvention displays the data in synchronization and sequentially, usersof the invention can visually witness the location of the suspect and inconjunction with video accounts of his actions at each location thatrecorded his actions through video surveillance. This form of datavisualization is particularly valuable for a jury when walking throughthe events of a suspect's activity during the incident time of a crime.

In a further embodiment of the present invention, the method of thepresent invention initializes a first multimedia file from a firstancillary physical entity. The multimedia file includes locationmetadata constituents in reference to the estimated location of a targetevent and metadata constituents that uniquely identify the target event.A second multimedia file, from a second ancillary physical entity isinitiated and includes metadata constituents that provide a detailedaccount of what occurred at the target entity and metadata constituentsthat uniquely identify the target event. In the event the uniquelyidentifying metadata constituents are equivalent between bothperspectives of the target event, the metadata constituents are resolvedinto a single event. Once rendered as a single target event, thelocation metadata constituents of the target entity can be visualized inconjunction with the metadata constituents providing a detailed accountwhat occurred of the target entity.

In a further embodiment of the present invention depicted in FIG. 6, themethod 300 of the present invention initializes a first multimedia filefrom a first physical entity. By multimedia file, it is meant a filethat upon execution represents a work of textual, audio, or visualnature. The multimedia file includes quantifiable first multimediametadata constituents. A quantifiable data element is any data that canbe expressed as a quantity or magnitude, for example 100 degreesCelsius, but not “hot.” An exemplary data category capable ofquantification include time indicia. A second multimedia file, from asecond physical entity is initiated 304 and includes quantifiable secondmultimedia metadata constituents. The multimedia files are pulled from afile database 394.

Quantifiable metadata constituents are compared 306 to ascertaincategorical metadata constituents. Categorical metadata constituents arethose data elements that are of the same category, for example velocitydata in a first multimedia file are categorically related to velocitydata in a second multimedia file, but time data would not becategorically related to temperature data. Whether data is of the samecategory may be determined definitionally rather than substantivelythrough the use of a relational glossary found in a relational database396. Metadata of the multimedia files may be normalized 308 as describedelsewhere in this document.

The domain of the categorical quantifiable metadata constituents isdetermined 316. By domain, it is meant the set of data values for whicha data category has values. For example, for a video of thirty secondslength, the domain may have values at each second from one to thirtyseconds; or more likely, depending on the granularity desired by theprogram, the domain may be the frame rate multiplied by the time span ofthe video. The domain may include (i) actual data element valuescaptured, (ii) data element values extrapolated/estimated topredetermined data values; (iii) a smaller set of data element valuesculled from a greater set of data element values; or (iv) otherwise.

The multimedia files, first and second, are rendered 310 insynchronization and sequentially on the visual display as a function of,and substantially equivalent to the domain of, any selection of saidmultiple quantifiable categorical metadata constituents. A selection ofmetadata constituents may be offered to the user of the present method300. A common means of qualitatively comparing multimedia files is basedon time; however, here the multimedia files may be rendered sequentiallyand in synchronization. The sequential nature of the present inventionprovides that the selected data type controls the rendered nature of themultimedia file. A multimedia file may be rendered pursuant to time, asis commonly done, or a multimedia file may be rendered based on, say,altitude. Because the domain of the altitude has been determined for thefirst multimedia file and the second multimedia file, the firstmultimedia file may be presented sequentially by altitude rather thantime. Because the metadata has been normalized, the multimedia files maybe rendered in synchronized fashion pursuant to any of the selectabledata element types of the multimedia files. Rather than playing a seriesof photographic stills by time, for example, such still may beprogressed by latitude, distance from an origin, geolocation, or other;and furthermore, the photographs from the vantage of two or moredisparate physical entities may be compared based on similar data valuesas synchronized based on the data value types. Furthermore, the sequencevalue of data, and the first multimedia file or portion thereof, relatedto the data value can be adjusted in synchrony with a data value of thesecond multimedia file. For example, in the descending drone example,multimedia files may be depicted based not on time or altitude, butinstead on humidity or pressure. In the event that multiple multimediafiles, or portions thereof, correspond to a particular data value,multiple windows may be utilized to depict the multiple files orportions thereof. The multimedia files may be adjusted 320 sequentiallyin forwards or reverse while remaining synchronized. In other words, ininstances where the multimedia files are sequenced according toaltitude, say from 0.0 m to 10.0 m, the present invention may displaymultimedia files from 2.0 m to 5.0 m sequentially by altitude whereinmultimedia files from two entities are rendered at the same time at thesame altitude. The present invention may scroll the altitudes in reverseor forwards with both altitudes being equal and displaying themultimedia files created at the altitudes displayed.

Although the present invention allows the files to be displayed insynchronization, this is predicated on the sequential equivalencebetween the files. To the extent that the domain of a first file runsfrom Time=0 s to Time=90 s, and a second file runs from Time=15 s toTime 30 s (and time is the data point chosen as the basis of thesequence), then the first file will be rendered from Time=0 s to Time=14s absent a rendering of the second file. From Time=15 s to Time=30 s,the files will be synchronized by the sequential order based on time.And then from Time=31 s to 90 s, the first file may be rendered absentthe second file. In other words the system and process are capable ofsynchronization at data points, but the invention is not tied to onlydisplay of synchronized points.

The method concludes 330 a solution based on the relationship betweenthe metadata constituents of the first multimedia file and the secondmultimedia file. A preferred example is to draw a geospatialrelationship between the first physical entity and the second physicalentity, as shown in FIG. 2. The relationship may extend beyondgeospatial data, including indicating any metadata constituentrelationships between the first physical entity the said second physicalentity. When a broad range of metadata constituents are considered, thepresent invention may expose previously unknown metadata constituentrelationships between said first physical entity and said secondphysical entity. A preferred concluded solution 330 is an output of twometadata constituents from the separate entities along with acalculation expressing the relationship between the metadataconstituents. Another conclusion 330 may indicate synchronization issuesin which categorical metadata constituents are inappropriately parallelin order to provide a means to modify in bulk the categorical metadataconstituents of the second physical entity to be appropriately parallelto the first physical entity, in which case a manual alteration 332 maybe applied for a final product or for reprocessing of data through oneof more steps of the process 300 of the present invention. A conclusion330 may indicate normalization issues, data integrity flaws, andincomplete data, wherein the alteration 332 step may provide a means tonormalize, correct, and augment metadata constituents of the firstphysical entity and the second physical entity. A conclusion 330 mayalert a user to a duplicate representation of the first physical entity,wherein the alteration 332 step may provide a means to resolve/winnowduplicate physical entity representations into a single physical entityrepresentation. A conclusion 330 may indicate a visual representationthat provides situational awareness and event understanding previouslyunavailable. The visual representation may include a visual overlay fora map, as described in other embodiments of the present invention and/orthe visual representation may spatially relate to the first entity orsecond entity in a way that provides information tied to the metadataconstituents. An example of a spatial relationship tied to an entity mayinclude a bomb blast radius or firearm line of sight or coloring indiciaof the entity in a color scheme gradient relating to danger, whereas anexample of a visual representation not tied to an entity includesproviding a color scheme alert merely in conjunction with the entity.The visual alert may directly relate to a constituent of may be derivedtherefrom; an example of the former may include a color gradient relatedto vehicle speed, an example of the latter may include a color gradientrelated to danger based on vehicle speed.

Other embodiments of the present invention may include multiple physicalentities bearing radioactivity sensors. Multimedia files bearingradioactivity metadata constituents may be sequentially rendered, andwhen prompted by user of the process, the process and system mayindicate a relationship between the metadata constituents at a momentselected by the user. For example, if the multimedia files includesvideos with frames displayed, not as a function of time, but as afunction of Curie reading, a user could halt the video display at aparticular point at which the system and process may inform the user ofthe distance between the physical entities or the acceleration thereof.

When a file is sequentially displayed as a function of time, there arerarely sequence problems. There is only one Jan. 1, 2015, 14:30:00 Zulu.However, when a multimedia file is arranged as a function of anon-unique element, e.g. pressure, multiple portions of the multimediafile may occupy a sequence space. To accommodate for this, the presentinvention should have a basis of raising or lowering granularity ofmeasurement, utilizing multiple windows for a single file whenappropriate, initiating high or low pass filters that block data beyonda particular range, or utilizing a secondary basis of sequentialarrangement. A secondary basis of sequential arrangement permits thepresent invention to arrange data primarily on the basis of its sequencefor a prime data category (e.g., pressure) and to permit a more linearsequence utilize a secondary data category as a tie breaker (e.g.,time). The second data category should preferably be one with moreunique values. To the extent that sequential times cannot be, or aredesired not to be, perfectly linearly, the present invention may decideto display multiple files for a single data point. For example, aweather balloon may have multiple occurrences of altitude of 50.0 m; andrather than attempting to fashion a linear arrangement out of a data setthat inherently lacks a linear data set (e.g., time), two or more of anentity's multimedia files related to its perception at that altitude maybe displayed for altitude=50.0 m.

Turning now to FIG. 8, the file manipulation engine 250 for manipulatingdata to conform efficiently to the present invention. A client 252 andserver 254 are in communication such that information 140, such asfiles, may be passed therebetween. The communication may be over a localarea network or wide area network—or other form of computer-to-computertransmission. Alternatively, a file may simply be present on memorylocal to the processor adapted to perform a process 200, 300 of thepresent invention. The file is ingested 256 are recognized as beingcapable of use with a process 200, 300 disclosed herein.

There are now a series of decisions to determine the nature of the file.A computation-efficient means of analyzing the data of a file is tosimply examine 258 the extension of the file. This could be problematic,as data within a file may not conform to the published parameters of theextension. The file may be evaluated 260 to determine whether the filecontents fit within a known file architecture from which the filemanipulation engine 250 may glean data for aggregation into a data setor whether data sets preformatted for the present invention arepre-existing on the file. The evaluator 260 may further form a decisionbased simply on the existence of a recognized extension.

Data is then passed to a data processor 262 that transforms 264 the datainto a common data model 266 that is used to compare like data types,insert pointers to executable files, and the like. The preferred commondata model is conceptually similar to the data model expressed in thefile 140 of FIG. 7.

FIGS. 9-10 depict a computer ecosystem 700 of the present invention. Byecosystem it is meant one or more computers 702 that areorganizationally related. The ecosystem may include computers undercommon ownership, computers that belong to the same network or series ofnetworks, computers that are collaborating, etc. The present inventionmay be provided as a computer program product, or software that mayinclude a computer-readable storage medium 704 having stored thereoninstructions, which may be used to perform the process of the presentinvention across a computer ecosystem 700 according to the variousembodiments disclosed herein.

A computer 702 of the present invention may include any combination ofone or more computer readable media 704. The computer readable mediummay be a computer readable signal medium or a computer readable storagemedium. A computer readable storage medium may be, for example, but notlimited to, an electronic, magnetic, optical, electromagnetic, infrared,or semiconductor system, apparatus, or device, or any suitablecombination of the foregoing. More specific examples (a non-exhaustivelist) of the computer readable storage medium would include thefollowing: an electrical connection having one or more wires, a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), an optical fiber, a portable compact disc read-onlymemory (CD-ROM), an optical storage device, a magnetic storage device,or any suitable combination of the foregoing. In the context of thisdocument, a computer readable storage medium 704 may be any tangiblemedium that can contain, or store a program for use by or in connectionwith an instruction execution system, apparatus, or device.

A computer readable signal medium 704 may include a propagated datasignal with computer readable program code embodied therein, forexample, in baseband or as part of a carrier wave. Such a propagatedsignal may take any of a variety of forms, including, but not limitedto, electro-magnetic, optical, or any suitable combination thereof. Acomputer readable signal medium may be any computer readable medium thatis not a computer readable storage medium and that can communicate,propagate, or transport a program for use by or in connection with aninstruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmittedusing any appropriate medium, including but not limited to wireless,wireline, optical fiber cable, RF, etc., or any suitable combination ofthe foregoing.

These computer program instructions may also be stored in a computerreadable medium that can direct a computer, other programmable dataprocessing apparatus, or other devices to function in a particularmanner, such that the instructions stored in the computer readablemedium produce an article of manufacture including instructions whichimplement the function/act specified in the flowchart and/or blockdiagram block or blocks.

The computer program instructions may also be loaded onto a computer,other programmable data processing apparatus, or other devices to causea series of operational steps to be performed on the computer, otherprogrammable apparatus or other devices to produce a computerimplemented process such that the instructions which execute on thecomputer or other programmable apparatus provide processes forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks.

The flowchart and block diagrams in the figures described belowillustrate the architecture, functionality, and operation of possibleimplementations of systems, methods, and computer program productsaccording to various embodiments of the present invention. In thisregard, each block in the flowchart or block diagrams may represent amodule, segment, or portion of code, which comprises one or moreexecutable instructions for implementing the specified logicalfunction(s). It should also be noted that, in some alternativeimplementations, the functions noted in the block may occur out of theorder noted in the figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. Furthermore, the functionality of one block maybe subsumed by the functionality of another block as a substep thereof.It will also be noted that each block of the block diagrams and/orflowchart illustration, and combinations of blocks in the block diagramsand/or flowchart illustration, can be implemented by special purposehardware-based systems that perform the specified functions or acts, orcombinations of special purpose hardware and computer instructions.

An ecosystem 700 may further include a computer network or data networkthat allows computers to exchange data. In a computer network of thepresent invention, networked computing devices pass data to each otheralong data connections. The connections between nodes are establishedusing cable media, wireless media, or other media. The Internet or otherexterior network 790 may be a component of the ecosystem 700. Nodes mayinclude hosts such as personal computers, phones, servers, andnetworking hardware. Two such devices are networked together when onedevice is able to exchange information with the other device, whether ornot they have a direct connection to each other. Computer networks ofthe present invention support applications such as access to the WorldWide Web, shared use of application and storage servers, printers, andfax machines, and use of email and instant messaging applications.Computer networks may be included irrespective of the physical mediaused to transmit their signals, the communications protocols to organizenetwork traffic, the network's size, topology, and organizationalintent.

It is preferred that the network of the present invention have at leastone boundary 720, and potentially multiple boundaries if a demilitarizedzone is utilized. The boundary 720 may include any number of layersdesigned to regulate and secure the flow of information betweennetworks. Boundary layers of the present invention may includeenterprise content management software, firewalls, filters, threatmanagement software, alarms, etc. Software for establishing a boundarymay be run on a server 710 with server storage 730 of the presentinvention, which may include directory services controlling accesscredentials.

To combat security risks posed by network connections, firewalls arefrequently used. A firewall may be a hardware or software component thatfilters network traffic so that communications with unauthorized thirdparties are blocked but legitimate network functions may be carried out.Frequently, the filters applied by a firewall are specified by a set ofpolicies defining characteristics of network messages that either shouldpass through the firewall or that should be blocked. Because differentlevels of communication may be appropriate depending on the origin ordestination of messages, firewall policies may be provided for eachapplication that executes on a computing device and communicates over anetwork.

A firewall may have an outward side facing a global network, such as theInternet. The opposite side of the firewall may be a private networkthat is protected by the firewall. The private network may include anynumber of host machines (e.g., computers) each addressable by its own IPaddress. The physical construction of the network may be such that alldata packets intended for one of the IP addresses behind the firewallpass through the firewall. Using the firewall rules, which may be set bya network administrator or other user, the firewall may determinewhether to allow or deny certain data packets and/or determine where toroute particular data packets based on the IP addresses to which thepackets are directed. The determination of where to route data packetsmay be done using the IP addresses of the host machines in the privatenetwork.

Depending on the addressing scheme used by the network, the IP addressesof the host machines may be static or dynamic. Static IP addresses donot change over time, and thus once they are set in the firewall rules,there is no need to update them. The Internet Protocol version Four(IPv4) addressing system commonly uses static addressing, while IPv6 mayuse dynamic addressing. Dynamic IP addresses may change over time andthus, there is a need to update the firewall rules as changes occur.When a small Local Area Network (LAN), such as a domestic network in aprivate residence, is linked to a larger network such as the Internet,the link is often through a gateway router acting as a firewall. One ofthe functions of the firewall is to protect the LAN from intrusion fromoutside.

A service directory accessible by a server 710, usually on serverstorage 730, stores information about network resources across a domain.An example of a directory service is Active Directory. The main purposeof Active Directory is to provide central authentication andauthorization services for Windows-based computers. Active Directoryalso allows administrators to assign policies, deploy software, andapply critical updates to an organization. Active Directory storesinformation and settings in a central database.

An Active Directory structure is a hierarchical framework of objects.The objects fall into three broad categories: resources (e.g. printers),services (e.g. e-mail) and users (e.g., user accounts and groups). TheActive Directory provides information on the objects, organizes theobjects, controls access and sets security. Certain objects can also becontainers of other objects. An object is uniquely identified by itsname and has a set of attributes—the characteristics and informationthat the object can contain—defined by a schema, which also determinesthe kind of objects that can be stored in the Active Directory.

Typically, the highest object in the hierarchy is the domain. The domaincan be further sub-divided into containers called Organizational Units.Organizational units give a semblance of structure to the organizationeither based on administrative structure or geographical structure. Theorganizational unit is the common level at which to apply grouppolicies, which are Active Directory objects themselves called GroupPolicy Objects. Policies can also be applied to individual objects orattributes as well as at the site level (i.e., one or more IP subnets).

The present invention may use one of more communication networks tofoster information exchange throughout the computers of the ecosystem.Communication networks might either be private or public. In a privatenetwork, communications between multiple computers occur in a secureenvironment that prevents access from outside the network withoutappropriate authentication. These networks are considered as “trusted”networks because the communication signals securely travel from onecomputer to another within the private network without being exposed tothe external environment.

Public networks such as the Internet, on the other hand, are not securebecause the communication over these networks is not private and issusceptible to interception by other computers. In addition, the publicnetworks cannot guarantee the delivery of the data packets being sent.They allow packets to be injected into, or ejected out of, the networksindiscriminately, and analyzed while in transit. To keep data sent overa public network private, a Virtual Private Network (VPN) is commonlyestablished on top of a public network when two computers use the publicnetwork to communicate with each other. In a Virtual Private Network,data sent from one computer to another is encrypted by a securitygateway and transmitted in encrypted form over the public network to asecond security gateway connected to the receiving computer. The secondgateway decrypts the data before forwarding it to the receivingcomputer. Such a private channel established on top of another networkis referred to as a network tunnel.

In order to set up a Virtual Private Network, a user first establishes apath to a VPN server and goes through an AAA process (Authentication,Authorization and Accounting) for identification and authorization tocreate a secure tunnel with the server. Once the user is authorized, asecure network tunnel is established between the user and the VPN serverover the public network, using a VPN protocol such as IPsec. Thisprocess requires a VPN client on the user's side, a VPN server and otherVPN hardware on the other side of the tunnel, as well as appropriateuser configurations.

Today's private networks often include wireless networks such as WiMAXto accommodate mobile access. In addition, to provide mobility access ina large geographic area, a private enterprise often relies onthird-party wireless infrastructures besides its own wireless network.In this case, a user's device would need to be authenticated by both athird-party gateway and an enterprise authentication server before itcould access the enterprise network. User credentials are typicallyrequested by and securely returned to the third-party gateway. Once theuser is authenticated and authorized, the user may communicate with thethird-party wireless gateway.

The present invention includes files 708, which may include executableinstructions by which the present invention runs, or files upon and withwhich the present invention interacts. The documents may be on localstorage 704 or shared storage 730 and be created, accessed, edited,and/or otherwise modified using any of a number of applications,including for example and without limitation Final Cut Pro, Avid,Microsoft Office applications (Word, Excel, Power Point, Outlook, Visio,etc.), Adobe Reader or Acrobat, AutoCAD, SolidWorks, or any othersuitable document editing application. The content of the documents maybe audio tracks, video clips, images, word processing documents,presentations, spreadsheets, business documents, engineering documents,databases, etc.

Although the present invention has been described in considerable detailwith reference to certain preferred versions thereof, other versionswould be readily apparent to those of ordinary skill in the art.Therefore, the spirit and scope of the appended claims should not belimited to the description of the preferred versions contained herein.

What is claimed is:
 1. An event visualization method comprising:initializing a first multimedia file, from a first physical entity,comprising quantifiable first multimedia metadata constituents,including time indicia; initializing a second multimedia file, from asecond physical entity, comprising quantifiable second multimediametadata constituents, including time indicia; comparing quantifiablemetadata constituents of said multimedia files to ascertain categoricalmetadata constituents normalizing said categorical quantifiable metadataconstituents; determining a domain of said categorical quantifiablemetadata constituents; rendering on a visual display said firstmultimedia file and said second multimedia file in synchronization andsequentially based on, and substantially equivalent to a time domain of,said time indicia of said first multimedia file and said secondmultimedia file; and indicating a conclusion based on metadataconstituents from said first physical entity and said second physicalentity.
 2. The method of claim 1 wherein said indicating step includesindicating a conclusion visually on said visual display, said conclusioncomprising a comprising a logical relationship between said firstphysical entity and said second physical entity.
 3. The method of claim1 wherein said indicating step includes indicating a conclusioncomprising synchronization issues in which categorical metadataconstituents are inappropriately parallel, and providing a corrector tomodify in bulk the categorical metadata constituents of said secondphysical entity to be appropriately parallel to said first physicalentity.
 4. The method of claim 1 wherein said indicating step includesindicating a conclusion comprising data compatibility selected from agroup consisting of: normalization issues, data integrity flaws,incomplete data sets, and combinations thereof of said first physicalentity and said second physical entity, and providing a correctionfunction selected from a group consisting of: normalizing, correcting,augmenting metadata constituents, and combinations thereof of said firstphysical entity and said second physical entity.
 5. The method of claim1 wherein said indicating step includes indicating a conclusioncomprising duplicate representation of at least one of said firstphysical entity and said second physical entity, and providing acorrector to resolve duplicate physical entity representations into aunified physical entity representation.
 6. The method of claim 1 furthercomprising the step of adjusting said rendering of said first multimediafile and said second multimedia file bi-directionally according to saidtime domain.
 7. An event visualization method comprising: initializing afirst multimedia file from a first physical entity comprising multiplequantifiable first multimedia metadata constituents; initializing asecond multimedia file from a second physical entity comprising multiplequantifiable second multimedia metadata constituents; comparingquantifiable metadata constituents of said multimedia files to ascertaincategorical quantifiable metadata constituents; normalizing saidcategorical quantifiable metadata constituents; determining a domain ofsaid categorical quantifiable metadata constituents; and rendering on avisual display said first multimedia file and said second multimediafile in synchronization based on a sequential rendition of said firstmultimedia file and said second multimedia file based on, andsubstantially equivalent to said domain of, any selection of saidmultiple quantifiable categorical metadata constituents; indicating aconclusion based on metadata constituents from said first physicalentity and said second physical entity.
 8. The method of claim 7 whereinsaid indicating step includes indicating a conclusion visually on saidvisual display, said conclusion comprising a comprising a logicalrelationship between said first physical entity and said second physicalentity.
 9. The method of claim 7 wherein said indicating step includesindicating a conclusion comprising synchronization issues in whichcategorical metadata constituents are inappropriately parallel, andproviding a corrector to modify in bulk the categorical metadataconstituents of said second physical entity to be appropriately parallelto said first physical entity.
 10. The method of claim 7 wherein saidindicating step includes indicating a conclusion comprising datacompatibility selected from a group consisting of: normalization issues,data integrity flaws, incomplete data sets, and combinations thereof ofsaid first physical entity and said second physical entity, andproviding a correction function selected from a group consisting of:normalizing, correcting, augmenting metadata constituents, andcombinations thereof of said first physical entity and said secondphysical entity.
 11. The method of claim 8 wherein said indicating stepincludes indicating a conclusion comprising duplicate representation ofat least one of said first physical entity and said second physicalentity, and providing a corrector to resolve duplicate physical entityrepresentations into a unified physical entity representation.
 12. Themethod of claim 8 further comprising the step of adjusting saidrendering of said first multimedia file and said second multimedia filebi-directionally according to said time domain.
 13. The method of claim8 further comprising the step of adjusting said rendering of said firstmultimedia file and said second multimedia file bi-directionallyaccording to said domain of at least one categorical quantifiablemetadata constituent.
 14. An event visualization system comprising: afirst physical entity exhibiting geospatial activity and with adetection array adapted to generate: (i) an entity location data setcomprising location data points bearing location metadata constituents,including geospatial indicia and time indicia, and (ii) a multimediafile from a vantage of said first physical entity comprising multimediametadata constituents; a second physical entity exhibiting geospatialactivity; a file manipulation engine bearing said first physical entitylocation data set and adapted to accept a representation of said secondentity geospatial activity and compare metadata constituents of saidlocation data points of said first physical entity to saidrepresentation of said geospatial activity of said second physicalentity; and a visual display adapted to output multimedia files fromsaid vantage of said first physical entity calculated to depict saidsecond physical entity.
 15. The system of claim 14 wherein said secondphysical entity includes: a second physical entity exhibiting geospatialactivity and with a second detection array adapted to generate: (i) anentity location data set comprising location data points bearinglocation metadata constituents, including geospatial indicia and timeindicia. and wherein said file manipulation engine is adapted to comparemetadata constituents of said location data points of said firstphysical entity to metadata constituents of said location data points ofsaid second physical entity.
 16. The system of claim 15 wherein saidsecond physical entity representation of said geospatial activityincludes said second entity location data set.
 17. An eventvisualization method comprising: initializing a location data set of afirst physical entity and a second physical entity, both entitiescomprising location data points bearing location metadata constituents,including geospatial indicia and time indicia; initializing a multimediafile from a vantage of said first physical entity comprising multimediametadata constituents; and comparing said second entity location datapoints to said first entity location data points to ascertain aninclusion of said second physical entity in said multimedia file; andrendering on a visual display said multimedia file having an inclusionof said second physical entity in said multimedia file.
 18. The methodof claim 17 further comprising the step of initializing a multimediafile from a vantage of said second physical entity comprising multimediametadata constituents; and wherein said comparing step include comparingmetadata constituents of said first physical entity location data pointsto metadata constituents of said multimedia file from said secondphysical entity to ascertain an inclusion of said first physical entityin said multimedia file of said second physical entity.
 19. The methodof claim 17 wherein said comparing step includes comparing said firstentity location data points as retained as metadata constituents of saidmultimedia file.